Convex Mathematical Programs for Relational Matching of Object Views

Automatic recognition of objects in images is a difficult and challenging task in computer vision which has been tackled in many different ways. Based on the powerful and widely used concept to represent objects and scenes as relational structures, the problem of graph matching, i.e. to find correspondences between two graphs is a part of the object recognition problem. Belonging to the field of combinatorial optimization graph matching is considered to be one of the most complex problems in computer vision: It is known to be NP-complete in the general case. In this thesis, two novel approaches to the graph matching problem are proposed and investigated. They are based on recent progress in the mathematical literature on convex programming. Starting out from describing the desired matchings by suitable objective functions in terms of binary variables, relaxations of combinatorial constraints and an adequate adaption of the objective function lead to continuous convex optimization problems which can be solved without parameter tuning and in polynomial time. A subsequent post-processing step results in feasible, sub-optimal combinatorial solutions to the original decision problem. In the first part of this thesis, the connection between specific graph-matching problems and the quadratic assignment problem is explored. In this case, the convex relaxation leads to a convex quadratic program , which is combined with a linear program for post-processing. Conditions under which the quadratic assignment representation is adequate from the computer vision point of view are investigated, along with attempts to relax these conditions by modifying the approach accordingly. The second part of this work focuses directly on the matching of subgraphs -- representing a model -- to a considerably larger scene graph. A bipartite matching is extended with a quadratic regularization term to take into account relations within each set of vertices. Based on this convex relaxation, post-processing and the application to computer vision are investigated and discussed. Numerical experiments reveal both the power and the limitations of the approach. For problems of sizes which occur in applications the approach is quite reasonable and often the combinatorial optimal solution is found. For larger instances the intrinsic combinatorial nature of the problem comes out and leads to sub-optimal solutions which, however, are still good

Radical Immersions: Navigating between virtual/physical environments and information bubbles - DRHA 2019 Conference Proceedings

This publication consists of the peer-reviewed papers and posters presented at the DRHA 2019 Conference "Radical Immersions: navigating between virtual / physical environments and information bubbles". The conference was held at Watermans Arts Centre, London (8-10 September 2019). For further information: http://www.2019.drha.uk

Probabilistic subgraph matching based on convex relaxation

Abstract. We present a novel approach to the matching of subgraphs for object recognition in computer vision. Feature similarities between object model and scene graph are complemented with a regularization term that measures differences of the relational structure. For the resulting quadratic integer program, a mathematically tight relaxation is derived by exploiting the degrees of freedom of the embedding space of positive semidefinite matrices. We show that the global minimum of the relaxed convex problem can be interpreted as probability distribution over the original space of matching matrices, providing a basis for efficiently sampling all close-to-optimal combinatorial matchings within the original solution space. As a result, the approach can even handle completely ambiguous situations, despite uniqueness of the relaxed convex problem. Exhaustive numerical experiments demonstrate the promising performance of the approach which – up to a single inevitable regularization parameter that weights feature similarity against structural similarity – is free of any further tuning parameters.

Data capture and real-time data quality analysis

This report presents results obtained in the CageReporter project regarding the development of a 3D vision system to be used for data capture in fish cages. The developed system enables to obtain high-quality data with the overall goal to identify fish conditions and perform cage inspections during daily operations, as well as the robotic vision for an underwater vehicle during the adaptive operation planning in the cage. A compact and robust sensor with optical components and lighting system was developed. In addition, this activity presents development of methods to evaluate the quality of the captured data. Based on defined quality criteria associated with fish conditions and cage inspection operations, algorithms have been developed to evaluate whether the quality criteria are met. The algorithms have been validated using image data obtained from 24/7 video streams from a full-scale fish cage. The work furthermore includes the development of image processing algorithms to estimate the distance and orientation relative to the inspected object of interest, such as the fish or the net. The developed algorithms have been validated based on vision data obtained during tests both in lab- and full scale.publishedVersio

Vision-based pose estimation for autonomous operations in aquacultural fish farms

There is a largely increasing demand for the usage of Unmanned Underwater Vehicles (UUVs) including Remotely Operated Vehicles (ROVs) for underwater aquaculture operations thereby minimizing the risks for diving accidents associated with such operations. ROVs are commonly used for short-distance inspection and intervention operations. Typically, these vehicles are human-operated and improving the sensing capabilities for visual scene interpretation will contribute significantly to achieve the desired higher degree of autonomy within ROV operations in such a challenging environment. In this paper we propose and investigate an approach enabling the underwater robot to measure its distance to the fishnet and to estimate its orientation with respect to the net. The computer vision based system exploits the 2D Fast Fourier Transform (FFT) for distance estimation from a camera to a regular net-structure in an aquaculture installation. The approach is evaluated in a simulation as well as demonstrated in real-world recordings

Vision-based pose estimation for autonomous operations in aquacultural fish farms

There is a largely increasing demand for the usage of Unmanned Underwater Vehicles (UUVs) including Remotely Operated Vehicles (ROVs) for underwater aquaculture operations thereby minimizing the risks for diving accidents associated with such operations. ROVs are commonly used for short-distance inspection and intervention operations. Typically, these vehicles are human-operated and improving the sensing capabilities for visual scene interpretation will contribute significantly to achieve the desired higher degree of autonomy within ROV operations in such a challenging environment. In this paper we propose and investigate an approach enabling the underwater robot to measure its distance to the fishnet and to estimate its orientation with respect to the net. The computer vision based system exploits the 2D Fast Fourier Transform (FFT) for distance estimation from a camera to a regular net-structure in an aquaculture installation. The approach is evaluated in a simulation as well as demonstrated in real-world recordings.publishedVersio

Performance Evaluation of a Convex Relaxation Approach to the Quadratic Assignment of Relational Object Views

We introduce a recently published convex relaxation approach for the quadratic assignment problem to the field of computer vision. Due to convexity, a favourable property of this approach is the absence of any tuning parameters and the computation of high–quality combinatorial solutions by solving a mathematically simple optimization problem. Furthermore, the relaxation step always computes a tight lower bound of the objective function and thus can additionally be used as an efficient subroutine of an exact search algorithm. We report the results of both established benchmark experiments from combinatorial mathematics and random ground-truth experiments using computer-generated graphs. For comparison, a recently published deterministic annealing approach is investigated as well. Both approaches show similarly good performance. In contrast to the convex ap- proach, however, the annealing approach yields no problem relaxation, and four parame- ters have to be tuned by hand for the annealing algorithm to become competitive

Post-processing and visualization techniques for isogeometric analysis results

Isogeometric Analysis (IGA) introduced in 2005 by Hughes et al. (2005) [1] exploits one mathematical basis representation for computer aided design (CAD), geometry and analysis during the entire engineering process. In this paper we extend this concept also for visualization. The presented post-processing and visualization techniques thereby strengthen the relation between geometry, analysis and visualization. This is achieved by facilitating the same mathematical function space used for geometry and analysis also for post-processing and visualization purposes. During non-linear analysis derivatives are incrementally computed and stored with different basis function representations. We introduce and investigate projection methods to be able to use the same function space for both displacements and stresses without loss of accuracy. To obtain a common representation for structured and unstructured meshes like hierarchical spline, locally refined B-spline (LR B-spline) and T-spline techniques we exploit Bézier decomposition in a post-processing step resulting in a Bézier element representation and constitute it as generalized representation. The typically used unrelated (fictitious) finite element mesh representation for visualization purposes are easily replaced without changing the underlying geometry as well as the algorithmic data structure. One further benefit of the used Bézier decomposition lies in the fact that it facilitates a natural parallel implementation on Graphics Processor Units (GPUs) exploiting shader programming. In this paper we have developed and investigated an accurate, efficient and practical post-processing pipeline for visualization of isogeometric analysis results. The proposed IGA visualization pipeline consists of three steps: (1) Projection, (2) Bézier decomposition and (3) Pixel-accurate rendering. We have tested four different projection methods. A description on how to perform Bézier decomposition of LR B-splines are given (whereas for hierarchical and T-splines this has been done before). Furthermore, the use of GPU shader programming to enable efficient and pixel-accurate visualization is detailed. The performance of the four different projection techniques has been tested on manufactured problems as well as on realistic benchmark problems. Furthermore, the IGA visualization pipeline has been demonstrated on a number of real-world applications